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EXECUTIVE  SUMMARY 


The  Performance  Measurement  Indicators  System  (PMIS)  being  developed  by  tfie  Naval  Sea  Systems  Command, 
Ordnance  Navy  Industrial  Fund  community  (ORD/NIF)  is  at  an  important  juncture,  lb  date  much  of  the  devebpment 
effort  has  focused  on  monitoring  reduction  goals  established  by  the  Naval  Industrial  Improvement  Program  (NIIP). 
Research  and  private  sector  experience  indicate  that  emphasis  on  reduction  programs  to  improve  performance  is 
at  best  a  short-term  or  tactical  solution  which  often  is  self-defeating.  This  point  is  made  by  many  noted  scholars 
of  management  such  as  Deming,  Drucker,  Kami,  Skinner,  Sink,  Walton,  Peters  and  Whterman. 

A  successful  performance  measurement  system  must  be  linked  to  the  long-term  strategy  of  an  organization 
and  not  to  near-term  tactics  according  to  experts  such  as  Sink.  Drucker  and  Deming  point  out  that  control  or 
performance  measurement  systems  are  significant  factors  in  foe  culture  of  an  organization.  There  is  increasing  evidence 
that  the  most  appropriate  culture  in  knowledge-based  organizations  is  that  of  commitment  rather  than  control.  The 
commitment  organization  approaches  performance  and  productivity  improvement  by  setting  high  performance 
standards  to  increase  the  effectiveness  or  output  of  its  resources.  Control  organizations  focus  on  efficiency  or  inputs 
and  set  minimum  acceptable  performance  standards.  Control  organizations  are  accordingly  slow  to  recognize  the 
real,  though  often  intangible,  assets  such  as  customer  satisfaction,  quality  consciousness  and  employee  motivation. 
Commitment  organizations  are  faced  with  the  challenge  of  measuring  and  accounting  of  these  intangible  assets. 

The  Centers  of  Excellence  strategy  of  the  ORD/NIF  is  a  major  step  toward  a  commitment  organization.  A 
second  major  step  would  be  a  performance  measurement  system  aligned  with  the  culture  and  desired  results  for 
the  Centers  of  Excellence.  Such  a  performance  measurement  system  would  give  the  ORD/NIF  community  a  tool 
to  focus  on  its  efficiency,  an  input  side  issue,  and,  more  importantly,  focus  on  its  effectiveness,  an  output/outcome 
side  issue.  This  would  help  the  Centers  of  Excellence  develop  the  necessary  expertise  and  resources  to  meet  the 
major  growth  in  fleet  support  requirements  driven  by  increased  numbers  of  ships  and  the  ewlution  of  more  technically 
sophisticated  weapon  systems. 

This  approach  conforms  closely  with  the  views  expressed  recently  by  Senator  Sam  Nunn  in  the  New  Y(xk 
Times... “'We  must  not  only  have  the  right  forces,  we  must  manage  those  forces  effectively  and  efficiently....  Over 
the  longer  term,  defease  managers  must  increase  military  capabilities  without  large  increases  in  defense  spending. 
This  will  require  stable,  predictable  levels  of  defense  spending  and  improving  the  quality  and  authority  of  senior 
civilian  acquisition  managers  in  the  Pentagon.”  Further  development  and  refinement  of  the  Operational  and  Global 
Indicators  System  is  essential  for  the  ORD/NIF  community  to  have  true  Centers  of  Excellence. 
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INTRODUCTION 


This  technical  rqtort  tq^praises  the  Naval  Sea  Systems  Command,  Ordnance  Navy  Industrial  Fund  (ORD/NIF) 
performance  measurement  ^tem  development  effort  led  by  the  Naval  Ordnance  Station,  Indian  Head,  MD 
(NAVORDSIA).  Performance  measurement  has  been  discussed  and  addressed  by  almost  every  oi^anization  in  the 
Department  of  Defense.  In  the  ORD/NIF  community,  tfadi  of  the  eleven  sites  have  to  a  degree  developed  their 
own  performance  or  productivity  measurement  system  during  the  past  several  years.  As  part  of  the  Naval  Industrial 
Improvement  Program  (NIIP),  a  formal  dfort  to  toild  a  Pe^rmance  Measuremmit  Indicators  System  (PMIS)  was 
undertaken  by  the  Na'ol.Sea.Systems  Command  »(NAVSEASYSCOM),‘  Combat  System  Field  Operations  and  Ord¬ 
nance  Support  Group'^(SEA-06G)p  NAVORDSTA  was  assigned  to  lead  the  PMIS  development  effort  under  the  coor¬ 
dination  of  Dr.  Dominic  J.  Monetta,  Technical  Director  of  NAVORDSTA. 


This  report  has  a  twjfold  purpose; 

Document  the  PMIS  development  effort, 

Suggest  areas  where  the  effort  could  be  improved.^ 

_ _ _  _ _  -  _  _ 

References  to  management  theory  and  private  sector  ©cperiences  based  on  research  conducted  by  Indian  Head 


highlight  and  support  the  concepts  presented  in  the  report. 
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OVERVIEW 


This  report  discusses  several  thought-provob'ng  points  regarding  the  NIIP  strata  to  improve  productivity  and 
recommends  an  approach  to  further  develop  ORD/NIF's  performance  measurement  system.  Here  is  a  summary 
of  the  points  discussed: 

•  The  cost  reduction  approach  to  productivity  improvement  is  at  best  a  short-term  solution. 

•  Short-term  solutions  are  detrimental  to  productivity  and  self-defeating  unless  tied  clearly  to  strategic  objectives. 

•  Public  sector  performance  improvement  efforts  are  frustrated  by  a  limited  understanding  of  output  and  par¬ 
tially  defined  and  unmeasured  objectives. 

•  Effective  performance  measurement  systems  and  productivity  improvement  programs  are  driven  by  an 
organization’s  mission,  culture  and  objectives. 

•  Successful  private  sector  companies  in  knowledge-based  industries  are  adopting  a  commitment  organiza¬ 
tion  focused  on  long-term  effectiveness,  value-added  involvement  and  ownership,  and  a  customer  orientation. 

These  points  have  significant  bearing  on  the  NIIP  objectives  and  the  strategic  thrust  of  the  ORD/NIF  commu¬ 
nity,  both  of  which  need  to  be  reflected  in  the  Performance  Measurement  Indicators  System.  The  recommended 
approach  is  to  further  develop  the  performance  measures,  especially  at  the  global  level,  to  build  on  the  “Centers 
of  Excellence”  by  linking  performance  with  the  objectives  and  desired  results  of  these  organizations.  This  approach 
will  move  the  ORD/NIF  community  closer  to  the  commitment  organizations  that  have  achieved  substantial  fun¬ 
damental  productivity  impiovement  similar  to  that  needed  to  support  the  fleet  and  respond  to  budget  pressures. 
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BACKGROUND 


The  initiative  to  develop  a  formal  system  for  performance  measurement  was  fundamentally  driven  by  die  NUP. 
An  important  point  to  bear  in  mind  is  diat  the  NIIP  itself  was  largely  a  response  to  developments  or  trends  in  upgrading 
the  effectiveness  of  the  fle^.  The  most  significant  developments  were  0)  the  major  expansion  of  the  fleet,  and 
(2)  modernization  of  the  fleet  with  sophisticated  high-performance  weapons,  surveillance,  and  operating  ^sterns. 

These  two  developments  within  the  Navy  to  enhance  effectiveness,  challenged  the  ORD/NIF  community  to 
fulfill  its  mission  to  maintain  fleet  readiness  without  increasing  its  resources  commensurate  with  the  growth  in 
demand  created  by  the  enhanced  and  expanded  fleet.  The  NAP  became  the  prime  strata  to  meet  the  challenge. 
The  NIIP  initiative  to  develop  a  formal  measurement  system  to  monitor  and  guide  performance  or  productivity 
improvements  is  a  critical  component  of  the  overall  program. 
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PERFORMANCE  MEASUREMENT  ACTION  TEAM  REVIEW 


One  of  the  ke^  action  items  of  the  Naval  Industrial  Improvement  Program  is  to  establish  a  formal  system  of 
performance  measurement  within  the  ORD/NIF  community.  In  June  of  1987  a  Performance  Measurement  Action 
Team  (PMAT)  was  assembled  under  the  chairmanship  of  Mr.  Myron  Holmes,  Director  of  the  Management  Analy¬ 
sis  Division  of  NAVORDSTA,  to  develop  such  a  system.  The  team  was  composed  of  representatives  from  field 
activities,  headquarters  and  an  outside  contractor  as  follows; 


M.  Holmes  (Chair) 
J.  O’Brien 
T.  Myers 
T.  Weaver 
E.  Hoar 

L.  Farner 

M.  Paten 
S.  Askew 
L.  Tbw 

J.  Johnson 
B.  Hoffman 


Naval  Ordnance  Station,  Indian  Head,  MD 
Naval  Ordnance  Station,  Indian  Head,  MD 

Naval  Ships  Weapons  Systems  Engineering  Station,  Pt.  Hueneme,  CA 

Naval  Weapons  Support  Center,  Crane,  IN 

Naval  Weapons  Station,  %rktown,  VA 

Naval  Undersea  Warfare  Engineering  Station,  Keyport,  WA 

Naval  Ordnance  Station,  Louisville,  KY 

Naval  Sea  Combat  Systems  Engineering  Station,  Norfolk,  VA 

Naval  Sea  Systems  Command,  Washington,  DC 

Naval  Sea  Systems  Command,  Washington,  DC 

Coopers  &  Lybrand,  Arlington,  VA. 


The  PMAT  established  the  following  objectives  for  a  performance  measurement  system  at  its  first  meeting 
in  early  July,  1987; 

•  Articulate  performance  up  the  line. 

•  Support  achievement  of  the  ORD/NIF  $300  million  saving  goal. 

•  Stimulate  performance  improvement  throughout  ORD/NIF  businesses. 

•  Provide  a  consistent  concept  for  performance  measurement  throughout  the  ORD/NIF  community. 


At  the  inaugural  meeting,  the  PMAT  agreed  foat  performance  indicators  should  be  customer-oriented  and  reflect 
corporate  values  from  the  top.  Indicators  should  ideally  be  available  at  the  level  where  the  work  is  performed  and 
be  consolidated  up  throu^  a  hierarchical  structure  to  provide  management  at  each  level  with  information  to  assess 
performance  trends.  Indicators  at  the  operational  or  working  level  make  it  easier  to  identify  and  implement 
improvements  to  each  organization’s  processes  for  planning,  production  and  services. 


Performance  indicators  should  focus  on  improvement  rather  than  standards  of  acceptability.  To  focus  on 
improvement  performance,  indicators  need  to  emphasize  time  series  data  to  allow  period-to-period  comparisons. 
The  team  felt  that  comparisons  of  organizations  to  one  another  carried  serious  risks,  given  their  differences  in 
mission  and  work  load.  They  concluded  that  such  comparisons  could  divert  energy  away  hrom  process  improve¬ 
ment  to  nonproductive  efforts  to  research  and  explain  to  management  l^itimate  differences  in  mission,  work  load, 
and  environment.  ORD/NIF  organizations  are  more  easily  measured  against  their  own  past  performance  rather 
than  the  performance  of  others. 
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Utilizing  die  Nominal  Group  Technique  (Delbecq,  1986),  the  PMAT  defined  five  categories  for  measuring 
performance  at  die  operational  level:  vKirkload,  productivity,  quality,  service,  and  cost  (output/cost).  These  eateries 
were  key  to  developing  performance  indicators  and  were  part  of  the  basis  for  reviewing  existing  measures  inside 
and  outside  the  ORD/NIF  community. 

The  team  recognized  that  developing  a  complete  hierarchical  ^tem  of  performance  indicators  throughout  the 
ORD/NIF  community  would  be  a  long-term  effort  and  felt  that  practical  global  indicators  needed  to  be  developed 
for  headquarters  to  be  better  informed  about  ORD/NIF  performance.  This  tvw>-level,  global  and  operational  approach 
to  performance  measurement  would  satisfy  die  needs  of  headquarters  and  the  stations  with  the  long-term  goal  to 
merge  or  roll  up  the  measures  from  bodi  levels  in  the  future. 

The  PMAT  reviewed  performance  measurement  efforts  inside  and  outside  of  the  ORD/NIF  community  to  identify 
sites  with  similar  concepts.  They  surveyed  each  of  the  ORD/NIF  stations  to  identify  performance  measures  cur¬ 
rently  being  used.  The  survey  of  the  stations  showed  that  performance  measurement  was  being  done  at  each  site, 
and  that  the  commanding  officers  and  senior  civilians  tended  to  be  the  prime  users;  i.e.  the  measures  were  generally 
global  indicators  of  the  station’s  performance.  Funct'onal  or  operational  measures  were  being  used  when  there  was 
a  clearly  defmable  output;  e.g.,  manufecturing  aaivities.  While  performance  measures  existed,  a  systematic  process 
to  review  the  measures  to  address  problems  and  improve  processes  was  not  well  defined. 

With  respect  to  performance  measurement  outside  the  ORD/NIF  community,  the  PMAT  selected  the  Public 
Works  Center  in  Norfolk,  VA  to  review  in  greater  detail  because  it  had  recently  implemented  a  performance  meas¬ 
urement  system  that  conformed  to  the  team’s  concept  for  operational  performance  measurement.  A  subgroup  visited 
the  Public  Works  Center  and  reported  the  following  findings,  particularly  with  respect  to  implementing  an  operational- 
level  performance  measurement  system: 

(1)  Major  cultural  and  business  behavior  changes  were  involved. 

(2)  Resistance  was  met  from  management  and  staff  in  implementing. 

(3)  “Top-down”  directicn  was  essential. 

(4)  A  folly  empowered  outside  agent  was  critical  to  drive  home  and  prevent  diversions. 

(5)  Program  implementation  included: 

•  Exemption  from  personnel  relations  in  order  to  have  the  flexibility  to  move  or  transfer  people  during 

startup. 

•  Mechanisms  for  experimentation  or  trials  to  test  and  document  productivity  and  efficiency  steps. 

(6)  Systematic  review  using  performance  measures  is  important  to  achieve  process  improvement. 

The  subgroup  also  identified  the  following  key  feamres  as  critical  to  a  performance  measurement  ^tem: 

•  Focus  on  customer  satisfaction 

•  Focus  on  mission  accomplishment 

•  Focus  on  performance  improvement 

•  Focus  on  overall  performance;  quality,  effectiveness,  cost  efficiency 

•  Measurement  plus  the  process  to  control  and  improve 

•  Clear  accountability. 

With  this  background  the  PMAT  decided  to  approach  the  station-level  indicators  as  operational  indicators  which 
would  supplement  existing  performance  indicators  used  at  the  stations.  Using  a  criteria-based  decision  process, 
the  PMAT  selected  pilot  sites  representing  the  major  businesses  of  ORD/NIF  in  order  to  facilitate  development. 
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The  measures  developed  and  tested  with  the  assistance  of  an  outside  implementor  at  these  sites  would  then  be  “rolled 
out”  to  sites  witii  similar  operations.  Having  developed  a  plan  for  the  field-level  indicators,  the  PM  AT  then  concen¬ 
trated  on  the  headquarters-level  indicators,  as  globals.  Tlie  initial  criteria  used  to  develop  these  measures  were: 

•  Existing  headquarter  (HQ)  objectives 

•  Perceived  HQ  concerns;  i.e.  “what  do  diey  want” 

•  Field  activities’  views  as  to  what  HQ  needed  or  should  be  told 

•  Survey  results. 

The  global  irfdicators  were  viewed  as  a  ti.  ’to  help  the  NAVSEA  Combat  Systems  Field  Operations  and  Ord¬ 
nance  Support  Group  (SEA-06G)  to  (1)  better  manage  field  activities,  (2)  hi^li^t  status  on  major  issues,  (3)  pro¬ 
mote  field  activities,  and  (4)  provide  accountability.  The  PMAT,  again  using  the  Nominal  Group  Te^nique,  considered 
approximately  80  indicators  which  were  consolidated  and  prioritized  to  25.  These  remaining  indicators  were  fur¬ 
ther  refined  and  augmented  several  times  by  sending  them  as  strawmen  to  the  stations  and  by  reviewing  them  with 
SEA-06G. 

On  15  September  1987  Dr.  Dominic  J.  Monetta,  as  a  member  of  the  ORD/NIF  Productivity  Steering  Council, 
briefed  the  ORD/NIF  Board  of  Directors  on  the  PMAT  approach  and  concepts  for  overall  perfbimance  measure¬ 
ment.  The  Board  gave  their  foil  support  to  the  approach  and  included  safety  and  security  as  additional  ar^  for 
global  indicator  development. 

In  mid-November  1987,  the  PMAT  proposed  to  implement  the  global  indicators  for  which  the  field  stations 
said  data  was  readily  available.  To  assure  feasibility  of  the  indicators,  all  the  global  indicators  were  first  to  be  tested 
using  NAVORDSTA  as  the  Beta  site.  The  results  of  the  test  were  presented  to  SEA-06G  in  February  1988,  and 
some  improvements  were  made  based  on  feedback  from  headquarters.  Appendix  A  provides  formats  with  sample 
data  for  these  ORD/NIF  performance  indicators.  Subsequently,  the  global  indicators  were  turned  over  to  an 
implementor  contractor  tasked  to  incorporate  global  indicators  into  an  overall  performance  measurement  program 
for  the  ORD/NIF  community. 
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PRIVATE  SECTOR  PARALLEL 


In  mai^  respects  die  challenge  confronting  the  ORD/NIF  is  similar  to  that  confronting  the  private  sector  in 
the  United  States.  In  both  cases  the  ultimate  requirement  is  for  higher  performance  to  satisfy  the  demands  of  cus¬ 
tomers,  shareholders,  and  employees.  The  influx  of  foreign  goods  and  services  has  been  driven  customers’  per- 
cqidons  of  quality  and  price  advantage.  Recent  and  currerit  business  restnicturiiig.  evidenced  by  mergers,  leveraged 
buy-outs,  ai^  acquisitions,  often  reflects  a  recognition  that  a  particular  business  is  not  performing  or  realizing  its 
potential  in  the  eyes  of  investors,  managers,  directors  or  shareholders.  Fundamental  to  these  competitiveness  or 
performance  issues  diced  by  the  private  sector  is  a  firm’s  ability  to  develop  and  implement  a  strategy  to  maximize 
the  effectiveness  of  resources  to  provide  high-quality  goods  and  services  at  a  competitive  price.  In  a  word,  the 
successful  firms  today  are  focusing  on  value  (Walton,  19S6). 

This  concept  of  value  is  evident  in  how  customers,  employees,  suppliers  and  shareholders  are  viewed  and  how 
products  and  services  are  made  and  delivered.  The  concept  of  value  has  both  strat^ic  and  operational  implica¬ 
tions;  the  concept  of  value  is  becoming  the  desired  culture  for  fostering  success.  Adopting  the  cotKept  frequently 
is  difficult,  however,  because  it  often  conflicts  with  traditional  management  practices  (Hackman,  1985).  The  differ¬ 
ences  in  the  approaches  are  implicit  in  their  descriptions.  The  traditional  approach  is  termed  a  “control”  riKidel 
in  which  work  is  broken  into  small  or  specialized  components  and  “performance  expectations  expressed  as  stan¬ 
dards  diat  defme  minimum  acceptable  performance”  (Walton,  1985a).  Within  the  management  structure  “inevitably 
layering  develops  and  is  justified  by  control  considerations  as  ‘...labor  is... managed  as  a  variable  cost’”  (Walton, 
1985a).  The  control  model  aligns  with  Frederick  W.  Tkylor’s  philosophy  from  the  early  part  of  the  twentieth  century. 

The  newer  model  developed  during  the  past  fifteen  years  emphasizes  commitment.  This  model  effectively 
accommodates  rapid  change  and  knowledge-based  activities  which  require  team  participation.  “PerformaiKe  ex¬ 
pectations  are  set  relatively  high;  thQ^  are  ‘stretch  objectives’,  rather  than  minimum  performance”  (Walton,  1985b). 
The  emphasis  is  on  upgrading  and  “continuous  improvement  ...oriented  to  the  marketplace...  radier  than  measure¬ 
ment  of  the  work  itself’  as  in  the  control  model. 

This  change  in  thrust  to  a  commitment-  or  value-driven  organization  has  created  an  accounting  lag  in  measur¬ 
ing  the  key  assets  or  attributes  of  such  an  organization.  Many  of  these  assets  are  intangibles,  such  as  customer 
satisfaction,  quality,  and  worker  motivation,  that  are  not  captured  or  recognized  by  today’s  accounting  methods 
even  thoup*’  the  benefits  are  real  and  growing  in  significance  at  a  rapid  rate  (Kaplan,  1985).  W.  Edwards  Deming 
alludes  to  this  problem  in  discussing  the  decline  of  U.S.  industry:  “By  1969,  the  comptroller  and  1^1  department 
began  to  take  charge  for  survival,  fighting  a  defensive  war,  baclo  to  the  wall.  The  comptroller  does  his  best,  using 
only  visible  figures,  trying  to  hold  the  compare  in  the  black,  unaware  of  the  importance  of  figures  that  are  unknown 
or  unknowable. .. .  Unfortunately  management  ly  the  comptroller  and  the  legal  department  brings  only  further  decline. 
Consequently,  the  traditional  and  most  basic  of  performance  measures,  the  bottom  line,  is  under  scrutiny  as  a  truly 
accurate  reflection  of  an  organization’s  performance  and  guide  to  future  investments”  (Deming,  1985). 
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Performance  Measurement— Strategic  Implications: 

Most  management  philosophies  share  the  importance  of  measuring  performance.  A  performance  measurement 
system  has  foree  key  functions:  (1)  monitor  or  control  operations;  (2)  assess  progress  or  improvements;  and  (3) 
higjhli^t  the  plans  and  expectations  of  an  organization  to  customers,  employees,  shareholders,  vendors,  etc.  (Sitik, 
1985;  Brady,  1984).  Management  theorists  and  pracdtioners  also  agree  that  an  organization's  performance  meas¬ 
urement  ^rstems  should  be  tied  directly  to  foe  organization’s  strat^c  goals  and  objectives.  As  stated  by  W.  Edwards 
Deming,  every  organization  must  “create  a  constancy  of  purpose  toward  improvement  of  product  and  service,  with 
the  aim  to  b^ome  competitive,  st^  in  business,  and  provide  jobs”  (Deming,  1985).  This  phrase  contains  both 
strat^ic  and  performance  connotations.  “Constancy  of  purpose”,  “product  and  service”,  “aim”,  “competition,” 
and  “jobs”  are  strat^k  references.  “Create",  “improvemwit”,  and  “provide”  are  references  to  performance.  Peter 
Drucker  emphasizes  foe  fundamental  impact  of  the  control  aspects  (rf  an  organization’s  performance  measurement 
system  on  its  strata  in  foe  following  quote  from  his  highly  acclaimeo  book  Management;  Thsks,  Re^asibilities, 
Practices: 


Yet  here  is  the  real  control  of  foe  institution,  foat  is,  foe  ground  of  behavior  and  the 
cause  of  action.  People  act  as  they  are  being  rewarded  or  punished.  For  this,  to  them, 
rightly  is  the  true  expression  of  foe  values  of  the  institution  and  of  its  true,  as  against 
its  professed,  purpose  and  role  (Drucker,  1974). 


What  Performance  to  Measure: 

Within  this  context  of  (1)  linking  an  organization’s  strategy  to  its  performance  measurement  system  and  (2) 
foe  features  of  a  control-  versus  a  value-  or  commitment-oriented  organization,  understanding  and  defining  what 
is  to  be  measured  becomes  critical.  A  generic  guide  to  areas  which  could  be  measured  prepared  by  Dr.  Scott  Sink, 
Director  of  foe  VPC,  a  center  studying  the  management  of  quality  and  productivity  at  Virginia  Polytechnic  Institute 
and  State  University,  is  presented  in  Figure  1  (Sink,  1985).  Of  particular  interest  are  foe  parallels  between  foe  different 
terminologies  used  by  Drucker,  Sink,  and  Peters  and  Waterman  (authors  of  In  Search  of  Excellence). 


DRUCKER 


SINK 


PETERS  &  WATERMAN 


OPERATING  BUDGET 
INNOVATION  - 


PROFITABILITY 
INNOVATION  - 


STICK  TO  KNITTING 
BIAS  FOR  ACTION 
CLOSE  TO  CUSTOMER 
HANDS-ON  VALUE  DRIVEN 
SIMPLE  FORM,  LEAN  STAFF 

PRODUCTIVITY  THROUGH 
PEOPLE 

AUTONOMY 


FIGURE  1.  ORGANIZATIONAL  SYSTEMS  PERFORMANCE  CRITERIA 
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Selection  of  the  generic  criteria  to  measure  is  guided  by  the  results  desired  by  the  organization.  Implicit  in 
the  selection  and  use  of  performance  measures  is  the  culture  of  die  organization  for  achieving  the  desired  results. 
The  key  determinants  in  selecting  what  to  measure  boil  down  to  two  foctors:  (1)  the  constancy  of  purpose,  i.e.  what 
does  the  organization  want  to  accomplish;  and  (2)  how  it  wants  to  cause  the  action  to  accomplish  its  purpose.  The 
measures  selected  send  a  clear  signal  to  an  oiganizadon’s  clients/customers,  employees,  suppliers,  etc.  The  com¬ 
munication  aspects  of  die  measures  are  important;  Sink  quotes  finom  Barnard  (1939)  to  emphasize  the  point;  “Essendal 
to  the  surviv^  of  an  organization  is  the  willingness  to  ccxiperate,  the  ability  to  communicate,  and  the  esustence 
and  acc^tance  of  purpose”  (Sink,  1985). 
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PERFORMANCE  MEASUREMENT  AND  THE  PUBUC  SECTOR 


Although  Dnicker,  Deming,  et  al.,  address  all  organizations  in  concept,  the  nature  of  the  public  sector  requires 
further  understanding  to  appreciate  the  sensitivities  and  difficulties  of  performance  measurement.  Defining  and 
measuring  output,  effectiveness,  productivity,  and  quality  and  innovation  in  terms  of  results,  not  activities,  for  .service 
organizations  is  a  struggle.  For  a  nonprofit  government^  entity  defining  and  measurmg  these  criteria  are  especially 
difficult  (Swiss,  1983).  Dnicker  highlights  the  tendeiK^  of  public  sector  organizations  “to  set  lofty  though  impre¬ 
cise  and  non-<piantifiable  objectives  as  one  of  the  Seven  Deadly  Sins  of  Public  Administrators”  (quoted  in  Ammons, 
1985).  lb  have  a  chance  at  performance,  a  program  needs  clear  targets,  the  attainment  of  which  can  be  measured, 
appraised,  or  at  least  Judged  (Ammons,  1985). 

Recognizing  die  uniqueness  of  the  public  sector  and  die  associated  constraints  is  the  first  step  in  avoiding  the 
frustration  which  plagues  performance  improvement  efforts  (McGowan,  1984).  Despite  the  difficulties  in  measur¬ 
ing  and  the  organizational  differences,  research  demonstrates  similarity  between  the  public  and  private  sectors  in 
defining  what  is  technically  important  in  selecting  performance  measures  (Bain,  1982;  Grizzle,  1985).  The  key 
technical  criteria  are  as  follows; 

(1)  Validity— does  it  accurately  reflect  what  is  to  be  measured? 

(2)  Completeness— does  it  take  into  account  all  components  of  the  area  to  be  measured? 

(3)  Comparability— does  it  allow  accurate  measuring  between  periods  or  organizations? 

(4)  Inclusive— does  it  tie  to  or  integrate  with  other  areas  measured  in  the  organization? 

(5)  Timeliness— does  the  measure  reflect  changes  soon  enough  for  taking  action? 

(6)  Cost  rffectiveness— does  the  measurement  effort  exceed  the  usefulness  of  the  measure? 


Where  Does  ORD/NIF  Fit: 

Several  issues  have  been  addressed  regarding  the  development  of  the  ORD/NIF’s  Performance  Measurement 
Indicators  System; 

•  The  background  which  prompted  the  need;  i.e.,  the  enhanced  effectiveness  of  the  fleet  and  the  ORD/NIF’s 
challenge 

•  A  review  of  the  PMAT  objectives  and  concepts  used  to  develop  a  performance  measurement  system  for 
the  ORD/NIF  community 

•  The  similarity  of  the  challenge  to  the  private  sector’s  competitiveness  challenge  and  how  the  private  sector 
has  responded  by  focusing  its  strategy  on  being  more  effective  and  creating  a  culture  of  value 

•  The  characteristics  of  the  traditional  control  and  the  relatively  new  commitment  organization  and  the  lag 
in  accounting  methods 
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•  The  role  of  performance  measurement  in  modem  management  and  why  it  is  critical  to  link  a  performance 
measurement  ^tem  to  an  otganization’s  desired  results  and  desired  culture 

•  An  overview  of  the  generic  categories  of  performance  measurement 

•  A  brief  review  of  foe  obstacles  in  measuring  performance  in  public  sector  and  v^at  technically  to  consider 
when  developing  performance  measures  in  either  foe  public  or  private  sector. 

The  PMAT  was  successful  in  laying  foe  conceptual  groundwork  for  PMIS  and  successful  in  getting  foe  ORD/NIF 
community  thinkii^  about  performance  measurement.  How  do  foe  PMAT  performance  measurement  concqpts  stand 
up  to  modem  management  theories?  What  are  foe  weaknesses?  How  can  fo^  be  improved? 

Conceptually,  PMAT  broke  the  PMIS  into  two  sets  cf  indicators,  global  and  operational.  This  af^roacfa  was 
designed  to  provide  sufficient  flexibility  to  the  stations  to  tailor  foe  operational  indicators  to  their  particular  needs. 
The  global  indicators  should  provide  uniform  key  performance  inh^ation  about  each  station  so  foat  SEA-06G 
can  monitor,  direct  and  refme  policy  and  strategies.  This  approach  is  generally  recommended  by  experts  in  foe 
field  of  performance  and  productivity  measurement  sudi  as  Sink,  Bain,  Thom  and  Kendrick  (Sink  19S5;  Bain, 
1982;  Thorn,  1983;  Kendrick.  1984). 

Kami's  research  highlights  the  need  for  three  types  of  controls  or  performance  measures.  The  first  are  opera¬ 
tional  controls  to  ensure  that  what  is  being  done  is  being  done  well.  The  second,  a  higher  level,  are  overall  controls 
designed  to  ensure  foat  what  is  being  done  is  appropriate  to  a  Ann’s  well-being  or  strat^.  Kami  cites  several  exam¬ 
ples  of  firms  which  had  sophisticated  operational  controls  for  micromanagement  but  ultimately  got  “out  of  con¬ 
trol...  because  major  issues,  organizational  entities,  projects  and  business  sectors  were  not  accounted  for.”  These 
firms  lacked  overall  controls.  The  third  control  that  Kami  emphasizes  is  a  qualitative  control  standard.  This  control 
is  in  contrast  to  the  quantitative,  especially  budget  and  financial,  nature  of  the  (^rational  and  overall  controls. 
Qualitative  controls  “get  behind  the  figures  and  measure  foe  quali^  of  foe  information  you  are  getting.”  In  many 
respects  qualitative  control  is  based  on  mechanisms  for  reviewing  performance  and  foe  “cross-flow  of  information 
below...  and  with  the  top  management  level”  (Kami  and  Ross,  1973). 

The  role  of  the  PMAT  in  getting  the  ORD/NIF  community  to  focus  on  performance  measurement  was  impor¬ 
tant  in  defining  foe  initial  global  indicators.  Ideally,  according  to  Sink  and  others,  a  particular  level  of  management 
should  use  seven,  plus  or  minus  two,  indicators.  The  list  of  global  indicators  below  should  be  viewed  primarily 
as  a  starting  point  for  refining. 

Global  Performance  Indicator  Subject  Areas 

Indirect  staffing 
Indirect  nonlabor  cost 
Inventory  investment 
Direct  labor  productivity 
Engineering  services  saving 
Total  NIIP  savings 
Manage  to  payroll 
Safety 
Security 

Activity  workload  by  COE/sponsor 
Workload  composite  by  COE 
Constrained  vs.  unconstrained  workyears 
Carryover  and  carryover  projection 


Performance  Measures 

EfficieiKy 

Efficiency 

Efficiency 

Efficiency 

Efficiency 

Efficiency 

Budgetability 

Quality  of  work  life 

(^ality  of  work  life 

Effectiveness 

Effectiveness 

Effectiveness 

Effectiveness 
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Net  operating  results 
ACP  obligation  rate 
Direct  labor  hours 
Facilities 


Budgetability 

Effectiveness 

Budgetability 

Effectiveness 


lb  more  closely  analyze  these  performance  measures,  a  review  of  the  criteria  or  eateries  listed  earlier  is 
helpful.  The  seven  noted  by  Sink  are  listed  and  descra)ed  (Sink,  1985). 

Q)  Effiectiveness— d^ee  to  which  desired  results  are  accomplished  on  time,  without  error  (quality),  in  the 
amount/magnitude  desired 

(2)  Efficiency— d^ee  to  which  resources  are  utilized,  especially  the  right  resources  in  the  rigfit  way,  versus 
a  plan  or  standard 

(3)  Quality— d^ee  of  conformance  to  customer  requirements;  fitness  for  use;  absence  of  errors  or  problems 
in  the  product  or  service  measured  at  five  checkpoints  in  the  system  to  ensure  total  quality  is  assessed 

(4)  Productivity— relationship  of  outputs  to  inputs  [Note:  This  measure  can  also  be  viewed  as  the  relationship 
of  effectiveness  to  efficiency  (Bain,  1982).] 

(5)  Quality  of  worklife— effective  response  of  employees  to  the  overall  work  environment;  e.g.,  secure,  safe, 
bored,  satisfied,  motivated,  and  so  forth 

(6)  Innovation— degree  to  which  there  is  creative  improvement,  technology  can  be  developed  and  assimilated, 
and  responsiveness  to  change 

(7)  Profitability/budgetability— relationships  between  fmancial  resources  and  their  use;  budget  performance. 

Adapting  these  seven  criteria  for  application  in  the  ORD/NIF  community  was  not  an  easy  process  despite  their 
apparent  straightforward  nature.  As  noted,  the  nonprofit  and  service  nature  of  many  government  entities  makes 
criteria  tied  to  ouqiut— namely  effectiveness,  productivity,  and  quality— hard  to  define  and  measure  in  terms  of 
actual  results.  The  global  indicators  when  analyzed  within  the  context  of  the  NIIP,  i.e.,  the  strat^ic  plan  for  meet¬ 
ing  the  challenge  to  support  an  expanded  fleet,  appear  to  emphasize  measurements  of  efficiency  and  budgetability. 

Recognizing  DOD’s  overall  budget  pressure,  die  emphasis  on  input  measures,  especially  budgetability,  is 
understandable.  Strat^ically  however,  these  measures  which  focus  on  inputs  are  flawed  because  they  do  not  follow 
the  emphasis  in  the  fleet  on  enhanced  effectiveness  or  increased  ouqiut.  This  flaw  should  be  resolved  or  it  will 
present  an  important  strat^ic  dilemma  to  the  ORD/NIF  community  in  the  long  term.  The  down-sizing  implicit 
in  emphasizing  efficiency  and  budgetability  as  an  organization’s  primary  performance  measures  reflects  a  basic 
change  in  the  strategy,  if  not  the  mission,  of  an  organization  (C^y,  Hegeman,  and  Shanks,  1987;  Applebaum, 
Simpson,  and  Shapiro,  1987).  When  the  demand  for  an  organization’s  ouqiut  is  increasing,  the  strat^ic  seriousness 
of  such  a  course  of  action  is  heightened. 

What  the  challenge  confronting  the  ORD/NIF  community  requires  is  improved  productivity;  i.e.,  get  more 
output  for  each  unit  of  input.  Within  this  context  emphasis  on  input-oriented  measures  is  questionable.  The  result 
required  is  improved  productivity  which  means  that  output  must  receive  at  least  an  equal  amount  of  emphasis  for 
the  performance  measures  selected  to  meet  the  validity  and  completeness  requirements.  These  are  the  two  most 
important  technical  criteria  (Bain,  1982;  Grizzle,  1985).  Without  an  emphasis  on  output,  the  measures  do  not  accurately 
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reflect  what  is  to  be  measured,  i.e.,  productivity,  from  the  validity  perspective.  Similarly,  without  consideration 
of  output,  die  measures  do  not  take  into  account  all  components  to  be  measured  and  therefore  do  not  meet  the 
completeness  requirements. 

Focus  on  input  is  a  common  approach  to  company  productivity  improvement  programs,  but  it  carries  substan¬ 
tial  long-term  risks  to  corporate  health.  Results  from  a  survey  conducted  by  Shetty  in  1983  and  1984,  which  included 
responses  from  171  major  U.S.  industrial  and  nonindustrial  Arms,  indicated  that  cost  reduaion  programs  were  the 
most  widely  used  strat^ies  for  improving  produaivity.  Below,  in  order  of  decreased  use,  is  a  summary  of  the 
results  obt^ed  by  Shetty  (Shetty,  1986): 

Cost  reduction 
Employee  participation 
Productivity  incentives 
Goal  setting  with  productivity  focus 
Increase  automation 
Quality  improvement 
Increase  employee  training 
Better  labor-management  relations 
Increase  research  and  development. 

According  to  Shetty,  fast  results  and  the  relative  ease  of  developing  and  implementing  were  the  principal  rea¬ 
sons  that  cost  reduction  programs  were  chosen  (Shetty,  1986).  Also  cited  was  the  economic  environment  of  the 
firm;  i.e.,  cost  reduction  programs  are  popular  when  sales  are  declining,  financial  strains  are  prevalent,  or  the 
firm’s  performance  is  poor  in  general.  Several  respondents  to  the  survey  noted,  however,  that  cost  reduction  pro¬ 
grams  should  be  viewed  only  as  a  short-term  effort  to  weather  a  storm.  Otherwise  respondents  commented  that 
the  firm  could  well  impair  its  long-term  health  and  wherewithal  to  sustain  productivity  growth.  “Continuous  cost 
cutting  may  fall  on  critical  activities  such  as  R  &  D,  market  development,  quality  control,  maintenance  and  rqiair, 
and  management  training,  whose  negative  impact  mjy  be  severe  in  the  long  run”  (Shetty,  1986). 

Deming  is  critical  of  short-term  thinking.  He  views  the  hope  for  quick  results  as  self-defeating  because  the 
“effect  in  the  long  run  erodes  investment  and  ends  up  as  just  the  opposite  to  what  is  intended”  (Deming,  1985). 
Skinner  shares  the  views  found  in  Shetty’s  research  and  advocated  by  Deming.  Skinner’s  studies,  primarily  of  manufac¬ 
turing  firms,  have  highlighted  that  a  “focus  on  cost  reduction  programs  is  proving  harmful.”  He  has  found  in  such 
circumstances  that  the  managers  “preoccupied  as  th^  are  with  this  week’s  cost  performance,  ‘resist  innovation’ 
as  they  know  well  that  changes  in  processes  or  systems  will  wreak  havoc  with  the  results  on  which  th^r  are  meas¬ 
ured.”  Skiimer  terms  the  situation  as  “The  Productivity  Paradox”  because  when  the  focus  is  solely  on  cost  reduc¬ 
tion  or  minimizing  inputs,  the  results  tend  to  be  only  a  “chipping  away  at  productivity”  (Skinner,  1986). 

Kami  views  situations  in  which  cost  cutting  actions  are  the  primary  response  to  productivity  pressures  as  “evi¬ 
dence  of  the  failure  or  lack  of  planning.  At  worst  it  is  indicative  of  ‘panic  management’,  an  affliction  that  affects 
many  corporations,  particularly  those  that  fail  to  establish  good  programs  of  planning  and  control....  This  approach 
can  do  irreparable  damage  to  personal  relationships,  established  systems,  and  employee  attitudes”  (Kami  and  Ross, 
1973). 

Productivity  paradox  and  crisis  management  are  not  complimentary  descriptions  of  strategies  which  place  primary 
focus  on  cost  reduction  programs  to  resolve  fundamental  productivity  issues  of  an  organization.  To  Skinner,  Dem¬ 
ing,  and  Kami,  the  best,  if  not  the  only,  way  to  approach  productivity  is  within  the  context  of  an  organization’s 
overall  goals  and  objectives.  Constancy  of  purpose,  which  nurtures  planning,  innovation,  and  unleashes  the  organi¬ 
zation,  is  the  first  requirement  in  effecting  significant  productivity  improvement  (Walton,  1986;  Deming,  1985; 
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Skinner,  1986).  The  need  for  a  performance  measurement  system  to  control,  monitor  and  guide  the  progress  is 
critical  as  (1)  it  becomes  the  primary  means  for  communicating  what  the  oiganization  desires  to  be  and  accomplish 
and  Q)  it  causes  the  action  as  it  provides  the  grounds  of  bdiavior  against  which  people  in  foe  oiganization  are 
rewarded  or  punished. 


ORD/NIF— Centers  of  Excellence  and  Perfomumce  Measurement: 

Hie  ORD/NIF  community  made  an  important  strategic  step  when  it  adopted  foe  Centers  of  Excellence  concept 
and  reorganized  accordingly.  One  of  foe  major  benefits  of  diis  step  was  to  reduce  foe  redundancy  caused  by  overly 
ping  skills  and  competition  for  resources  and  markets  within  foe  community.  Enhanced  focus  and  commitment 
are  equally  important  benefits  of  foe  Center  of  Excellence  strata  as  they  foster  true  understanding  of  a  site’s  busi¬ 
ness  and  objectives.  The  Center  of  Excellence  strata  is  consistent  with  the  commitment  model  of  oiganization 
preferable  for  knowledge-based  activities  or  businesses  like  the  ORD/NIF  community. 

The  commitment  model  organization  is  oriented  toward  productivity  gains  via  enhanced  effectiveness  or  out¬ 
put  from  its  resources.  Performance  objectives  are  set  consistent  with  a  long-term  goal  and  corresponding  culture 
of  greater  value  for  customers,  employees,  contractors  and  shareholders  and  so  forth.  The  ORD/NIF  community 
has  taken  steps  to  strategically  position  and  structure  itself  around  Centers  of  Excellence  to  respond  to  foe  fleet’s 
trend  toward  greater  effectiveness.  The  development  of  a  performance  measurement  system  to  cause  foe  action, 
i.e.,  to  guide,  monitor  and  control  foe  community,  is  an  important  next  stq>.  The  commitment  model  suggests  building 
a  critical  mass  and  supporting  that  organization  via  investing  in  it  so  that  continuous  productivity  gains  can  be 
achieved  by  realizing  more  effectiveness  from  available  resources;  i.e.,  more  output  per  unit  of  input.  Further 
development  of  the  ORD/NIF’s  performance  measurement  indicators  needs  to  be  done  in  foe  context  of  foe  long¬ 
term  Centers  of  Excellence  strategy  to  promote  continuous  improvement  at  both  foe  global  and  operational  levels. 
If  approached  in  this  manner,  research  and  experience  show  that  foe  measures  developed  (1)  are  more  likely  to 
be  technically  sound  as  desired  results  or  outputs  will  be  better  defmed  and  tied  to  inputs,  and  (2)  the  productivity 
improvements  will  be  longer  lasting  as  foe  cause  for  action  and  grounds  for  behavior  will  be  aligned  with  the  long¬ 
term  strategy  and  mission  of  foe  ORD/NIF  community. 
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CONCLUSION 


The  ORD/NIF  community  could  benefit  greatly  from  a  formal  ^tem  to  measure  performance.  The  efforts 
of  the  PMAT  have  increased  recognition  of  the  potential  benefits.  The  important  question  now  is  how  die  community 
wants  to  use  a  performance  measurement  system. 

A  system  that  emphasizes  input  or  efficiency  parameters  and  n^lects  strong  ties  to  a  strategic  plan  is  at  best 
a  quick-fix  ^proach.  This  approach  fails  to  sustain  improved  performaiKe  and  oftmi  has  native  long-term  conse¬ 
quences. 

Research  and  current  trends  in  management  practices  would  fovor  a  system  that  focuses  on  commitment  to 
and  continuous  improvement  in  the  effectiveness  of  or  output  from  resources  to  enhance  performance  and  produc¬ 
tivity.  The  ORD/MF  community  has  a  Centers  of  Excellence  strata  upon  which  it  can  build  a  commitment 
organization.  A  formal  performance  measurement  system  aligned  closely  with  the  Centers  of  Excellence  strata 
would  be  mutually  reinforcing.  Performance  measures  need  to  be  tied  to  to  more  clearly  defined  ouqiuts  and  out¬ 
comes  at  both  die  operational  and  global  levels.  The  performance  measurement  system  could  provide  the  centers 
and  SEA-06G  with  a  powerful  tool  to  monitor,  control,  coimnunicate,  and  create  die  desired  culture  and  actions 
to  achieve  results  and  meet  the  challenge  confronting  the  community. 

The  next  step  in  the  PMIS  development  effort  is  to  link  the  strategies  and  objectives  of  die  ORD/NIF  with 
performance  indicators.  This  step  will  require  a  thorou^  assessment  of  the  ORD/NIF  customers’  output  needs, 
such  as  enhanced  product  value,  military  utility,  and  productivity,  and  how  the  ORD/NIF  community  will  measure 
progress  and  fulfill  those  needs  in  the  near  and  long  term.  This  assessment  should  be  done  from  both  the  SEA-06G 
and  Centers  of  Excellence  perspectives.  Meshing  customers  assessments  with  die  ORD/NIF’s  strata  and  higher 
level  objectives  will  lead  to  definition  of  ouqiuts  and  effectiveness  in  specific  business  areas.  Only  dien  can  consis¬ 
tent  and  meaningful  performance  indicators  tied  to  output  and  effectiveness  be  develc^ed  to  ^temadcally  guide, 
monitor,  and  control  all  levels  of  the  ORD/NIF.  This  approach  links  performance  measures  to  output  and  customer 
needs  and  establishes  the  capability  duou^out  the  organization  to  measure  conformance  and  achieve  continuous 
improvement.  With  performance  measurement  linked  to  strata,  output  and  effectiveness,  the  ORD/NIF  will  be 
re^y  to  transition  fiom  a  control  to  a  conunitment  organization  which  it  must  be  to  successfully  support  an  ex¬ 
panded  fleet  and  more  sophisticated  wetqion  ^tems  in  die  environment  described  by  Senator  Nunn  (Nuim,  1988). 
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Appendix  A 

SAMPLE  ORD/NIF  PERFORMANCE  INDICATOR  DATA 


INDIRECT 

MANPOWER 


Comments: 


figure  a-2.  indirect  nonlabor 


FIGURE  A-3.  INVENTORY  INVESTMENT 
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FIGURE  A-5.  DIRECT  LABOR  PRODUCTIVITY  (RELATED  TO  NIIP  GOALS) 
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FIGURE  A-6.  DIRECT  LABOR  PRODUCTIVITY  (RELATED  TO  NIIP  GOALS) 
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FIGURE  A-7.  ENGINEERING  SERVICES  SAVINGS  (NIIP  GOAL) 


FIGURE  A-8.  TOTAL  NIIP  SAVINGS 


FIGURE  A-9.  MANAGE  TO  PAYROLL 
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i-n.  SAFETY  INDICATORS 


VIOLATION  RATE 


FIGURE  A-I3.  SECURITY 


INSPECTION  FINDING  RATE 


Comments:  No  Physical  Security  Inspections  this  FY  to  date.  Six 

Information  Security  inspections,  with  no  findings. 
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FIGURE  A-15.  ACTIVITY  WORKYEARS  BY  COE/SPONSOR 
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FIGURE  A- 16.  WORKLOAD  COMPOSITE  BY  COE 


FIGURE  A-17.  CONSTRAINED  VERSUS  UNCONSTRAINED  WORKYEARS 
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FIGURE  A-19.  NET  OPERATING  RESUm 


figure  A-21.  direct  labor  hours 


NIF  -  MAim^NANCE  AND  REPAIR 


FIGURE  A-22.  FACILITIES 


MAINTENANCE  AND  REPAIR 


FIGURE  A-23.  FACIUnES 
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